1. Field of the Invention
The present invention relates to a sign language translation system and method for recognizing a type of sign language and translating it into a spoken language.
2. Description of the Related Art
As described in Proceedings of Conference on Human Factors in Computing Systems CHI'91 (1991), pp. 237 to 242, a conventional sign language translation method recognizes a sign language by converting the motion of hands wearing gloves with special sensors into electrical signals and checking whether the pattern of signals matches any one of previously registered signal patterns.
Another conventional method described in JP-A- 2-144675 recognizes finger spellings by taking an image of a pair of colored gloves by a TV camera to derive color information of differently colored areas of the gloves and by determining whether the pattern of the derived color information matches any one of previously registered color information patterns.
There are three types of Japanese sign languages, including (1) "a traditional type sign language" having been used by auditory-impaired persons, (2) "a simultaneous type sign language" in which each sign language word is assigned a Japanese word, and (3) "an intermediate type sign language" in which each sign language word is arranged in the same order as a spoken language. The intermediate type sign language is most frequently used. With this type of sign language, sign words are given in the same order as a spoken language. Consider for example a Japanese sentence " , , , , , "
equivalent to a phonetic sentence "Watakushi wa fuyu ni hokkaido e ikou to omou" and corresponding to the English sentence "I am thinking of going to Hokkaido in the winter". In this case, only important words (also called independent words) such as a verb " "="iku"="go" and nouns " "="hokkaido" (Japanese district name), and " "="fuyu"="winter", are expressed in the sign language, and other words are generally omitted such as postpositions " , , , ,"="te, ni, wo, ha"="various words or word elements similar to English particles but postpositional according to the Japanese Grammar", words dependent on auxiliary verbs " , , , "="reru, rareru, you, tai"="various words or word elements used in a corresponding manner in English), and pseudo nouns " , , "="koto, mono, no"="various words used in a corresponding manner in English". In addition, the conjugations of conjugative words such as verbs, auxiliary verbs, adjectives, adjective verbs, are generally omitted. Accordingly, the first-mentioned conventional method which simply arranges sign language words in the order recognized, is difficult to express a spoken language.
The sign language mainly used in the U.S.A. is the American Sign Language (ASL). In the U.S.A, the "traditional type sign language" is widely accepted. Also in the case of ASL, articles such as "a, an, the" and prepositions such as "in, at, on, of, by, from" are often omitted.
The second-mentioned conventional method recognizes Japanese finger spellings " "="a", " "="i", " "="u" and so on, corresponding to English alphabet characters. Finger spellings are used as an alternative means for the case when sign words cannot be remembered or understood, and so conversation using only finger spellings is rarely had. This conventional method is not satisfactory for conversation, because each word in a sentence is expressed by giving the finger spellings of all characters or letters of the word.
Such problems of sign language recognition are associated not only with Japanese sign languages but also with other sign languages for English and other foreign languages. Such problems are inherent to the sign language which provides communications by changing the positions and directions of hands (inclusive of fingers, backs, palms), elbows, and arms, and by using the whole body including the face, chest, abdomen, and the like.